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specifiques concernant un ou plusieurs personnages du 
programme televise diffuse (130). Les informations 
supplementaires sont synchronises avec la composante 
audio du programme televise de sorte qu'il n'y ait pas 
conflit, et elles sont communiquees au client (101) via 
rinterface de reseau (1 10). Un client peut, a travers un 
reseau d'ordinateurs (1150), demander et recevoir des 
informations audio/video supplementaires concernant un 
programme video. Dans un autre mode de realisation, les 
informations audio/video supplementaires peuvent etre 
extraites d'un CD-ROM, ou d'un dispositif de stockage 
similaire (1164). Le programme video peut Stre, par 
exemple, un programme televise retransmis en direct ou 
un enregistrement magn£tique decate dans le temps d'un 
programme de television. Les informations 
.supplementaires sont synchronises avec le programme 
video au moyen d'un code temporel int£gr6 dans le 
programme video, ce qui permet de voir et/ou d'ecouter 
les informations supplementaires de maniere harmonisee 
avec les evenements pu actions du programme vid6o. 



characters within the broadcast television program (1 30), 
The supplemental information is synchronized to the 
audio component of the television program so that there 
is no conflict, and is communicated to the customer (101) i . 
via the network interface (110). Through a computer 
network (1150), a customer (101) can request and 
receive specific supplemental audio/video information 
related to a video program. Alternatively, the 
supplemental audio/video information can be retrieved 
from a CD-ROM, or similar storage device (1 164). The 
video program can be, for example, a live broadcast 
television program or a time-shifted tape recording of a 
television program. The supplemental information is 
synchronized to the video program by using a time code 
which is integrated with the video program. This enables 
the supplemental information to be viewed and/or 
listened io in harmony with the Events or action of the 
video program. 
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(57) Abstract 

An apparatus and method for providing supplemental information related to video programs 
is disclosed. Through a telephone or electronic network interface (110), a customer (101) can 
request specific supplemental information related to one or more characters within the broadcast 
television program (130). The supplemental information is synchronized to the audio component of 
the television program so that there is no conflict, and is communicated to the customer (101) via 
the network interface (110). Through a computer network (1 150), a customer (101) can request and 
receive specific supplemental audio/video information related to a video program. Alternatively, 
the supplemental audio/video information can be retrieved from a CD-ROM, or similar storage 
device (1 164). The video program can be, for example, a live broadcast television program or a 
time-shifted tape recording of a television program. The supplemental information is synchronized 
to the video program by using a time code which is integrated with the video program. This 
enables the supplemental information to be viewed and/or listened to in harmony with the Events 
or action of the video program. 
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SYSTEM AND METHOD FOR SUPPLYING SUPPLEMENTAL AUDIO AND 
VISUAL INFORMATION FOR VIDEO PROGRAMS 

BACKGROUND OF THE INVENTION 

5 

This 'invention relates to the field of video and 
broadcast television programs. More particularly/ the 
present invention provides a system and a method for 
television viewers to receive additional dialog and 
10 information, via a selected interface, related to 

characters on a specific broadcast television program and 
to receive supplemental information, via a computer data 
communication network, that is synchronized to the action 
and events of a video program. 

15 

Many television (TV) programs, such as daytime soap 
operas and situation comedies, are regularly watched by 
large audiences. Considering the growing number of TV 
sets in homes and the amount of time the average person 
20 spends watching TV, the size of the audience is expected 
to increase over time. Correspondingly, programs need to 
change and evolve to suit the ever-more diverse tastes 
and interests of this expanding audience. 

25 Television networks, cable companies and independent TV 
program providers offer a myriad of TV program categories 
to capture the interest of viewers. These TV program 
categories include, for example, game shows, talk shows, 
situation comedies, sporting events, and soap operas. 

30 The ultimate goal is to provide interesting programming, 
in order to capture a large segment of the viewing 
audience . 

One popular category of TV programming is the daytime 
35 soap opera. This type of program revolves around the 
day-to-day lives of characters featured within the 
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program. A plot or story- line for a typical soap opera 
can span several episodes, possibly an entire TV season. 
Each episode continues and develops the story-line 
further, and also develops the personalities of 
5 characters within the soap opera in an ever -expanding and 
detailed manner. 

Soap operas, as well as other TV programs, often develop 
a core group of passionate and dedicated viewers . Many 

10 viewers religiously watch every episode of their favorite 
TV programs loathing to miss even a single episode. 
With these viewers in mind, several magazines on the 
market summarize daily or weekly episodes of television 
programs for people who may have missed a particular 

15 episode. There are also 900 telephone numbers available 
that offer similar services for a fee. These are just 
two areas in which peripheral services have developed in 
response to TV viewer 1 s demand. 

20 Moreover, many TV viewers are extremely involved in their 
favorite TV program's story-line and characters. The 
writers of TV programs must weave complex and interesting 
plots to maintain the viewer's loyalty and interest in 
the program. In most instances, multi- faceted and well- 

25 developed characters are essential to the popularity (and 
longevity) of the program. Thus, a typical script for a 
TV program may be written, revised and edited numerous 
times to ensure that the result is interesting, 
realistic, believable or humorous. 

30 

Writers and producers spend a great deal of time 
developing characters and story- lines for TV programs 
that ultimately are not used in the broadcast version of 
the TV program. Many times several scripts, each with 
35 different plot twists and dialogue, are developed for 
each TV program episode. This is occasionally done to 
preserve the secrecy of the final broadcast episode. 

2 
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However, more often than not, these changes and alternate 
scripts are never broadcasted. Thus, the TV program 
producers and developers have no outlet to display this 
additional work-product (or by-product, as the case may 
5 be) to the viewing public. 

To a certain extent, dedicated viewers of, for example, a 
soap opera, live vicariously through the trials and 
tribulations of the TV characters. Any additional 
10 information on their favorite TV program is of interest 
to these viewers, particularly, the thoughts, feelings 
and motivations that help define the TV characters. 

However, the typical TV program format makes it difficult 
15 to develop the thoughts and feelings for a TV character 
in great detail. The story-line in a TV program must 
flow in a relatively quick and fluid manner. Unlike a 
novel, where an author can, between exchanges of 
dialogue, devote extensive time to a given characters 
20 thoughts, motivations, and considerations, the dialogue 
in TV programs must flow in a real-time manner. In many 
situations, this means that the writers must briefly 
summarize the thoughts or motivations of a character to 
keep the action flowing smoothly. 

25 

For example, a soap opera might have the lead female 
character saying to the lead male character: "I am going 
to New York for lunch. Would you like to join me?" 
However, her thoughts, which are not being broadcasted, 
30 may have her contemplating several devious things she is 
considering doing while in New York. These devious 
machinations may or may not occur depending upon the road 
the writers decide to follow for the story- line. 

35 A wealth of non-broadcast material (from story- line 
changes and revisions) could be created and used to 
supplement the audio tracks of TV programs. In the 
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example discussed above, a parallel audio track could be 
used to provide the additional thoughts of the lead 
female character to listeners who are simultaneously 
watching the TV broadcast. 

5 In this regard, modern stereo televisions can receive a 
secondary audio channel (SAP) . SAP technology has been 
used to provide descriptive video (DVS) , which prints 
words on the television screen describing the action for 
the deaf. SAP has also been used to provide non-English 

10 audio tracks for the TV programs or sporting events, as 
well as a running commentary on TV movies. Some radio 
stations also broadcast audio tracks of live events (e.g. 
sporting events) or TV programs. 

15 However, these conventional simulcasting systems provide 
everyone using the system with the same audio track; 
i.e., all viewers (or listeners) using these systems 
receive the same information. There is no way for a TV 
viewer to select individual information, for example, 

20 additional dialogue or thoughts of a particular character 
on a TV soap opera program. Moreover, these systems are 
typically provided for a monthly or annuai service fee or 
are provided at no cost, because charging on a per-TV 
program or per-use basis is administratively difficult to 

25 process, or not possible at all. 

Other systems have been developed using community antenna 
television facilities (CATV) that allow subscribers to 
interactively request still-television video frames with 
30 an accompanying audio message. However, the TV video and 
accompanying audio message provided by these systems are 
not related to, or synchronized to, broadcast TV 
programs. 

35 Phone services (e.g. chat rooms and conference calling 
systems) also exist that allow callers to dial -in and 
listen to pre-recorded audio messages or even listen to 

4 
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live events in progress. When a caller uses a dial in 
service to listen to a sporting event, for example, which 
is being simultaneously broadcasted on TV, the radio 
audio track replaces the TV audio track. Thus, these 
5 services are not synchronized to the TV program's audio 
track and are not designed to supplement the dialogue for 
the TV program. 

Until now, producers and broadcasters of TV programs have 
10 not been able to benefit from non-broadcast material 
developed in relation to a TV program's story-line or 
characters. Therefore, synchronized audio tracks 
providing supplemental information for TV programs have 
not been created. This supplemental audio information 
15 would be of great interest to devoted fans of TV 

programs. In addition, TV program broadcasters would 
have an alternative venue to bolster their TV program's 
characters, thereby creating even greater interest in the 
particular TV. program. Accordingly, there is a need for 
20 a system that solves the above described problems. In 

particular, a system that permits TV producers to provide 
additional valuable entertainment content to interested 
viewers. 

25 Additionally, many video cassette programs (e.g., rented 
movies) are regularly watched by large audiences. 
Considering the growing number of TV sets and video 
cassette recorders (VCR) in homes and the amount of time 
the average person spends watching TV, the size of the 

30 audience is expected to increase over time. 

Correspondingly, programs need to change and evolve to 
suit the ever-more diverse tastes and interests of this 
expanding audience. 

35 In recent years, the Internet has received a great deal 
of attention. The Internet is a vast and expanding 
network of computers and other devices linked together by 

5 
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various media, enabling all the computers to exchange and 
share data. The Internet provides an enormous amount of 
information about a myriad of corporations, products and 
services/ as well as educational, research and 
5 entertainment information. It is estimated that more 
than 30 million people worldwide now use the Internet 
with 100 million predicted to be on the "net" within a 
few years. 

10 One way to establish a presence on the Internet is by 
placing a web site on a host operating a web server 
within a given domain name (e.g., "USPTO.gov"). A web 
site consists of one or more web pages, which are 
basically computer data files. A web page is identified 

15 by a unique Internet address commonly referred to. as a 
Uniform Resource Locator ("URL") . When. the web server 
receives an inquiry from the Internet, it returns the web 
page data to the computer making the inquiry. The web 
page may comprise multiple pages of information including 

20 messages, hyperlinks (i.e., automatic links that permit 
jumps from one web page to another) , audio and graphical 
information, and combinations thereof. 

In this regard, some advertisers and producers of 
25 television programs, movies and video programs have 

created web sites to promote and to provide additional 
information concerning their respective programs or 
projects. For example, an Internet user can access a web 
site related to his favorite TV show (e.g., 
30 "http://www.spe.sony.com/ pictures/tv/seinf eld. htm" ) to 
get an episode schedule, get more information about the 
TV show's characters and actors, and find out about up- 
coming special events. Web sites have also been created 
which allow Internet users to gather additional 
35 information about products or services that are 
advertised in television commercials. 
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Television advertisers having associated web sites often 
display the Internet address, or URL of an associated web 
site during television advertisements. With the URL, a 
prospective customer can access the advertiser's web site 
5 to receive additional information about the advertiser, 
the advertised product and/or related products. 

As these web sites make clear, advertisers, producers and 
developers of video programs have the potential to 

10 provide the viewer with far more information than can be 
practically accommodated by conventional television or 
video cassette programs. For example, the typical TV 
program format makes it difficult to provide intricate 
details concerning the plot or characters within the 

15 program, because the story-line must flow in a relatively 
quick and fluid manner. Unlike a novel, where an author 
can devote extensive time to a given character, item, or 
event, the dialogue in TV programs must flow in a real- 
time manner. In many situations, this means that the 

20 writers must provide only a brief summary of a character, 
item or event to fit within the limitations of a 
television program and keep the action flowing smoothly. 

In contrast to broadcast television or video cassette 
25 programs in which playing time is premium, a web site 

providing supplemental information is not so constrained. 
In this regard, a system known as HyperTV has been 
disclosed which enables television viewers to view web 
pages that are launched at the time of a broadcast 
30 television program. This system allows a viewer to 
select additional information related to a television 
program at the time of its original broadcast . 

Another system, Intercast, has been developed which 
35 integrates content of a web page with a broadcast TV 
signal using the vertical blanking interval ("VBI") of 
the television signal. This system utilizes an 

7 
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undisplayed portion of the received signal to transmit 
data to a television receiver. Like HyperTV, this system 
enables a viewer to view the content of a web site 
related to the particular program at the time of the 
5 broadcast . 

Although these systems provide additional content to a 
viewer of a broadcast television program, they have 
significant shortcomings. Presently, these systems 

10 provide additional information related to the television 
program, such as live chat lines, on-demand audio and 
video clips, and online shopping, but these systems fail 
to provide supplemental information synchronized with the 
action and events occurring within a particular video 

15 program. 

In addition, the Intercast system requires sophisticated 
video decoding circuitry to extract transmitted data from 
the VBI . The Intercast system further requires circuitry 
20 to allow the user to interface with the television 

receiver and send viewer requests upstream to a service 
provider. 

Further, both Intercast and HyperTV require the viewer to 
25 view the television program at the time of the original 
broadcast to receive synchronized information. A viewer 
wishing to time-shift his viewing of a television 
program, for example, using a VCR, cannot receive 
synchronized additional information from any of the prior 
30 art systems. 

The failure of these system to provide synchronized 
supplemental information is a major disadvantage. To 
more fully enhance the viewer's experience, the 
35 supplemental information discussed above must be provided 
at the proper times to correspond with the action and 
events of the video program. 

8 
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A wealth of non-broadcast material (or non- video- taped 
material) could be created and used to supplement 
conventional video programs. In the example discussed 
5 above, additional audio and visual tracks could be used 
to provide the supplemental information desired by the 
viewer. Although, this supplemental information must be 
synchronized to the action and events occurring in the 
video program so that the viewer can receive this 
10 information at the proper times. 

SUMMARY OF THE INVENTION 

The present invention solves the foregoing deficiencies 
15 by providing a system and a method for enabling 

television program viewers to receive supplemental 
information related to the television program. 

The present invention advantageously enables television 
20 program viewers to select supplemental information 

related to particular characters in a television program 
through a telephone network interface . The selected 
supplemental information is then received by the viewer 
via an appropriate interface. 

25 

In particular, one aspect of the present invention is 
directed to a method of supplying supplemental audio 
information that is synchronized to a broadcast 
television program so that the supplemental audio 

30 information is not in conflict with the audio component 
of the television program. This method includes 
providing selection and synchronization information 
through the broadcast television program, and supplying 
the supplemental audio information in accordance 

35 therewith. 



9 
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The present invention also solves the foregoing 
deficiencies by providing a system and a method for 
enabling video program viewers to receive synchronized 
supplemental information related to video programs. The 
5 present invention advantageously enables video program 
viewers to request and receive the synchronized 
supplemental information through a computer data 
communication network interface. 

10 In particular, one aspect of the present invention is 

directed to a method of supplying supplemental audio and 
visual information that is synchronized to a video 
program. This method includes providing program 
identification and synchronization information integrated 

15 with the video program. The viewer makes a request for 
the supplemental information through a computer data 
communication network. The viewer provides the program 
identification and synchronization information integrated 
with the video program being viewed. The viewer then 

20 receives the requested supplemental information 

synchronized to the video program through the computer 
data communication network. This method allows the 
supplemental information to be synchronized to a live 
broadcast television program or a pre-recorded video tape 

25 program, such as a rented movie or a time-shifted 
playback of a television program. 

Another aspect of the present invention allows the viewer 
to interactively change the supplemental information as 
30 it is being received. This embodiment allows the viewer 
to make additional requests at certain times throughout 
the video program to, for example, skip or modify certain 
portions of the supplemental information. 

35 The present invention also provides an apparatus which 
integrates the video program and the supplemental 
information on the same monitor. In this aspect, the 

10 
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program identification and synchronization information 
are automatically extracted from the video signal, which 
eliminates the need for the viewer to provide this 
information separately. 

5 

It is an advantage of the present invention to provide 
synchronized supplemental information on a web site. 
Such information would be of great interest to devoted 
fans of TV programs, for example, who would like more 
10 detail about the characters, items and events within a 
program, and not just additional, general information. 

It is an additional advantage of the present invention to 
provide a system which increases the amount of 
15 information available about a video program, thereby 
creating even greater interest in a particular TV or 
video program. 

It is a further advantage of the present invention to 
enable advertisers to provide additional information on 

20 products promoted during a video program. In addition, 
the present invention enables an advertiser who could not 
otherwise afford television commercial time a means for 
affiliating with its product with a video program by 
advertising on a synchronized web site. Such a web site 

25 synchronized to a video program can provide significant 
additional advertising revenues to broadcasters and video 
program producers not available using prior art systems. 

Other aspects of the present invention are described 
30 below. 

BRIEF DESCRIPTION OF THE DRAWINGS 

35 FIG. 1 is a block diagram overview of one system 

constructed in accordance with the present invention. 
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FIG. 2 is a block diagram of the service controller of 
FIG. 1. 

FIG. 3 is a table depicting the network database within 
5 the service controller of FIG. 2. 

FIG. 4 is a table depicting the program database within 
the service controller of FIG. 2. 

10 FIG. 5 is a table depicting the character database within 
the service controller of FIG. 2. 

FIG. 6 is a table depicting the recording database within 
the service controller of FIG. 2. 

15 

FIG. 7 is a table depicting the billing database within 
the service controller of FIG. 2. 

FIG. 8 is a flow chart of the operation of the system. 

20 

FIGS. 9a and 9b are flow charts of the operation of the 
service controller. 

FIGS. 10a and 10b are flow charts depicting how the 
25 service controller processes information received from 
the IVRU. 

FIG. 11 is a schematic block diagram overview of one 
system constructed in accordance with the present 
30 invention. 

FIG. 12 is a schematic block diagram illustrating the 
program identification arid synchronization information 
displayed during the video program. 

35 
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FIG. 13 is a schematic block diagram illustrating the 
preferred integrated display device of the present 
invention. 

5 FIG. \4 is a flow chart illustrating a preferred method 
of presenting supplemental information in accordance with 
the integrated display device of FIG. 13. 

FIG. 15 is a table depicting the simulcast event database 
10 within the integrated display device of FIG. 13. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

15 In a preferred embodiment of the present invention, 

writers. for a TV program write supplemental dialogue for 
specific characters featured in the TV program. This 
supplemental dialogue consists of "thoughts" and 
"comments" not spoken aloud during the broadcast version 

20 of the TV program. For example, these thoughts could 
include a hidden agenda, a character's true feelings 
toward another character, or other motivational factors 
driving a character's actions. While many devoted fans 
are interested in this type of information, it is not 

25 easily conveyed to the viewing audience during the real- 
time exchange of dialogue between characters on the TV 
program. 

In the present invention, audio recordings of these 
30 thoughts and comments are recorded using actors from the 
TV program. Of course, voice -doubles for the actors 
could also be used to make the audio recordings. The 
audio recordings are then edited and synchronized to the 
. broadcast TV program. This process ensures that the 
35 additional thoughts and comments do not overlap the 
dialogue being spoken on the broadcast version audio 
track. These thoughts and comments are interspersed 

13 
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during the non- speaking periods and other appropriate 
periods in the TV program so that they can be listened to 
simultaneously with the TV program without conflict. 

5 In alternative embodiments, the supplemental audio 
information can be used for other purposes. Several 
examples of other types pf supplemental audio information 
for TV programs include: adult content and themes not 
available on the regular program, and multiple audio 

10 tracks of the same TV program, which provide different 
sets of audio information each time the TV program is 
viewed (and listened to). For example, each viewing could 
offer the hidden thoughts of a different character, or 
offer the viewer clues (e.g., hints concerning 

15 alternative actions or events that may or may not occur) . 
This may give the viewer the sense that he is somehow 
participating in the show. Supplemental audio information 
can also be used to provide additional content, also 
referred to as "hyper- content , " for news- type programs, 

20 consisting of information not included in the broadcast 
(e.g. an unabridged news report) . 

Traditional methods of broadcasting audio tracks, such as 
SAP or radio, are not suitable for the present 

25 application. The supplemental audio information must be 
made available only to customers requesting the service 
(i.e. not broadcasted to the world). In addition, there 
may be multiple versions of supplemental audio 
information for a particular TV program, so that 

30 different customers can choose different supplemental 
audio information for the same TV program, or watch the 
same TV program multiple times with different 
supplemental audio information each time. The 
supplemental audio information is synchronized with the 

35 audio track of the broadcast TV program - enhancing the 
audio track rather than merely replacing it . 
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In a further embodiment of the present invention, writers 
and developers of video programs prepare supplemental 
information related to TV show episodes, movies, video 
tape presentations, and the like. This supplemental 

5 information comprising additional audio, text, and visual 
information related to the video program is made 
available at an Internet web site accessible by the 
viewers of the video program. Viewers (i.e. customers) 
can simultaneously view the video program on a TV 

10 receiver, for example, and receive the supplemental 
information on their computer monitor display. 

To illustrate the utility of the present invention 
consider a situation where the writers and/or director of 

15 a murder mystery television program might incorporate a 
g suicide noteg in a particular scene. Due to time 
constraints, they may be required to quickly flash or 
briefly summarize the "suicide note" to the viewing 
audience. If a viewer were able to, however, he may wish 

20 instead to slowly peruse the "suicide note" left by the 
victim. This might give the viewer additional insight 
into a character or event of the program, or provide 
additional entertainment by allowing the viewer to 
speculate as to future events in the program. 

25 

In another situation, a particular scene within a video 
or television program may have multiple characters 
involved in separate conversations, although, only one 
conversation can be showcased for the viewers at a time. 
30 However, rather than focusing on the particular 

conversation being showcased at that moment, a viewer may 
wish to eavesdrop on the other conversations or get a 
different visual perspective in order to get a better 
look at or understanding of other characters . 

35 

The present invention enables the producers of these 
video programs to offer a wide variety of additional 

15 
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information about the characters, story- lines and other 
aspects of their video program while being less hampered 
by the inherent constraints of a video program such as 
time, story development and continuity. A few of the 
5 possible forms of supplemental information that can be 
supplied to the viewing public are shown by the following 
examples: 

■> 

Close-ups of specific items: As discussed above, a 
10 viewer may wish to slowly read and examine a "suicide 
note 11 left by a character in a murder mystery video 
program. Viewers of a medical drama may wish to view 
the medical records or test results, which are not shown 
to the viewers, of a patient being examined by a doctor 
15 on the show. Advertisers, sponsors and producers may 
provide close-ups of props along with related 
information, such as the history of the prop in the 
program or sales information related to purchasing the 
prop. 

20 

Alternative camera angles: This enables the viewer to 
focus on areas of the video program that are in the 
background or that are only quickly scanned in the video 
program. For example, during a legal drama, viewers may 
25 wish to focus on the defendants or jury's reaction to 
certain evidence or testimony, rather than an overview 
camera angle of the courtroom. 

Alternative scenes and dialogue: Viewers can enhance 
30 enjoyment of their favorite TV programs by following 

their favorite characters throughout the show. When, for 
example, that character leaves the scene to go to a new 
location, the viewer may elect to receive supplemental 
visual and audio information related to that -character's 
35 action in the new location which is not seen by the 
conventional viewing public. Further, supplemental 
visual and audio information such as out-takes, edited 
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scenes and alternative dialog may be provided to enhance 
the viewing experience . 

Interactive games: Viewers may engage in games of skill 
5 and trivia related to the particular video program being 
viewed. For example, online tournaments may be conducted 
whereby viewers may compete against one another during a 
broadcast video program. It should be understood that 
the Internet (or commercial online service provider such 

10 as America Online) can provide a back channel allowing 
the web site to collect and evaluate viewers 1 answers to 
determine a winner. The tournament may include questions 
testing a viewer's knowledge of past programs, questions 
relating to events occurring during the current program 

15 or questions requiring a viewer to predict future events 
during the current program. 

Other types of supplemental information: A running 
version of the video program's script may be available 

20 enabling a viewer to review the text of jokes or parts of 
dialogue that may have been missed. Mature content and 
themes not available on regular TV programs may be 
provided. Supplemental information can also be used to 
provide additional content, also referred to as "hyper- 

25 content," for news-type programs, consisting of 

information not included in the broadcast (e.g., an 
unabridged news report or background information) . Such 
supplemental material may be made available specifically 
during commercial interruptions in the video program to 

30 extend the time constraints of the video program. 

Supplemental information related to commercials or 
advertisements: Viewers can receive detailed information 
concerning the products or services being advertised 
35 within the video program. This may also include 
receiving answers to specific questions about the 
product, for example, where is the nearest dealer, or how 

17 
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much does a particular option cost. The viewers may also 
be able to receive coupons related to the products and 
services, as well as enter sweepstakes related to the 
advertisements. In addition, the synchronized web pages 
5 may provide links to other web pages (i.e. sponsor, 

producer or network) or links to real-time video stream 
commercials. 

In the present invention, the supplemental information is 
10 edited and synchronized to the action and events of the 
video program. This process ensures that the 
supplemental information is available to the viewer at 
the proper times throughout the video program. For 
example, as discussed above, when a doctor, in a video 
15 program is viewing a patient's medical records, this 

information is also made available to the viewer via the 
synchronized web site. Throughout the video program, the 
viewer receives supplemental images, audio, dialogue, 
scenes and information synchronized to the specific 
20 actions and events occurring within the video program. 
This is in contrast to conventional web sites related to 
TV shows, for example, which only provide general 
information and are not synchronized to the TV show, even 
if accessed while viewing the TV show. 

25 

FIG. 1 shows an overall system block diagram of a 
preferred embodiment of the present invention. In this 
embodiment, a customer 101 receives service information 

30 131, which is displayed discretely during the broadcast 
of a television program 130. The service information 131 
comprises a 900 telephone number and a program 
identification code. The program identification code is 
a ten digit alphanumeric code comprising selection 

35 information and synchronization information (none of 

which are shown) . The service information 131 is used by 
the customer 101 to request specific supplemental audio 

18 



CA 02283940 1999-09-13 



WO 98/43406 PCT/US98/05112 

information related to the television program 130, as 
discussed below. 

It is understood that the television program 130 is one 
5 of many television programs that are broadcasted or 

transmitted to the public. The television program 130 ii 
received and displayed by conventional electronic 
equipment (i.e. a television set, not shown) located in 
close proximity to the customer 101. The electronic 
10 display equipment may receive the television program 130 
through an antenna adapted to receive TV signals from a 
TV broadcast station, from a satellite transponder, a 
trunk cable from a CATV (i.e. cable TV) system, or from 
any other suitable transmission means. 

15 

The service information 131 is recorded on the same 
recording medium as the television program 130. For 
example, a television program can be recorded and then 
the service information 131 can be added during editing 
20 of the recorded television program. The service 
information 131 can also be added at the time of 
broadcasting the television program 130. 

It is also understood that the service information 131 
25 would be displayed even if the television program 130 is 
recorded on a recorded medium and played-back by the 
customer 101. While one customer 101 is depicted in FIG, 
1 receiving the service information 131 from the 
television program 13 0, it is understood that the 
30 television program 130 is broadcasted to any number of 
customers simultaneously. 

As illustrated in FIG 1., the customer 101, by utilizing 
his telephone (not shown) , communicates through a public 
35 switch telephone network 110 (PSTN) and an interactive 
voice response unit, IVRU 112, to a service controller 
200. The customer 101 provides the selection information 
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and synchronization information, to the service 
controller 200. In response, the service controller 200 
sends the supplemental audio information to the customer 
101. The communication process via the telephone is 
5 discussed in detail below. 

FIG. 2 is a block diagram of the preferred service 
controller 200. The service controller 200 includes a 
CPU 221 which performs the processing functions. The 

10 service controller 200 also includes a read only memory 
222 (ROM) and a random access memory 223 (RAM). The ROM 
222 is used to store at least some of the program 
instructions that are to be executed by the CPU 221, such 
as portions of the operating system or basic input -output 

15 system (BIOS), and the RAM 223 is used for temporary 
storage of data. A clock circuit 224 provides a clock 
signal which is required by the CPU 221. 

The CPU 221 can also store information to, and read 
20 information from, a data storage device 226. The data 
storage device 226 includes a network database 300, a 
program database 400, a character database 500, a 
recording database 600, and a billing database 700. 
These databases are described below. In addition, the 
25 data storage device 226 includes instructions (not shown) 
which can be read by and executed by the CPU 221, thereby 
enabling the CPU 221 to process requests. While FIG. 2 
depicts separate databases, a single database that 
incorporates the functions of the databases mentioned 
30 above can also be used. Additional databases may be 

added as needed to store a variety of other information 
that may be required for other purposes. 

The data storage device 226 also contains all the 
35 supplemental audio information recorded for each 
character, the supplemental audio archive 228 as 
described in more detail subsequently. Any of a wide 
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variety of storage mediums are suitable for this purpose, 
including, for example, audio cassettes, digital audio 
tapes (DAT), laser discs, and the like. 

5 In an alternative embodiment, the service controller 200 
may have an external interface for inputting supplemental 
information, either live or pre-recorded. This 
supplemental information may include, for example, live 
audio information or even video information coupled with 

10 an audio component. The live audio information can 

include, for example, alternative color commentary for 
sporting events (e.g., Howard Stern, a popular radio disc 
jockey, providing supplemental color commentary on the 
Superbowl), or even specific athletes wearing 

15 microphones/transmitters during the sporting contest 
(e.g., a customer 101 could dial 1-900-RANGERS to hear 
what their favorite hockey player is saying during the ' 
game) 

20 The service controller 200 also includes a communication 
port 225 connected to an inter-active voice response unit 
interface 212. The communication port 225 enables the 
CPU 221 to communicate with external devices. An IVRU 
112, as shown in FIG. 1, is linked to the service 

25 controller 200 via the IVRU interface 212. The IVRU 112 
is connected to the PSTN 110. 

A plurality of telephone trunks (not shown) terminate at 
the IVRU 112 to allow multiple callers (i.e., customers) 
30 to call the IVRU 112 simultaneously. These telephone 
trunks may be analog or digital . The IVRU 112 may also 
terminate North American standard digital signal one 
(1.544 Mb/sec) facilities in a manner similar to a 
private -branch exchange (PBX) . 

35 

The IVRU 112 allows the customer 101 to communicate with 
the service controller 200 using his or her telephone 
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(not shown) . Voice command prompts guide the customer 
101 through various menu options allowing the customer 
101 to communicate and obtain information from the 
service controller 200. 

5 

FIG. 3 is a pictorial representation depicting the 
information stored in the network database 300. The 
network database 300 comprises information related to 
each TV program provider or distributor associated with 

10 the present invention. The network database 300 includes 
a network name field 302, a network identification number 
field 304, and a number of participating programs field 
306. The network identification number field 304 is a 
unique two digit code corresponding to each network or TV 

15 program provider. While only four records R1-R4 for four 
networks are depicted in FIG. 3, any number of records 
may be stored. It is also understood that independent 
and syndicated broadcast program providers may also be 
included in this database. 

20 

FIG. 4 is a pictorial representation of the content of 
the program database 400. The program database 4 00 
comprises information related to each television program 
130 for which supplemental information is available. The 

25 program database 400 includes a program name field 402, a 
program identification number field 404 , a network 
identification number field 406. (corresponding to field 
304 of the network database 300), a broadcast times field 
408, a participating characters field 410, and a phone 

30 number field 412. The program identification number 
field 404 is a unique four digit code corresponding to 
each TV program. The participating characters field 410 
contains the names of various characters for which 
supplemental audio information is available. The phone 

35 number field 41 includes a unique telephone number 

associated with each television program 130. This unique 
telephone number is dialed by the customer 101 to request 
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supplemental audio information related to that particular 
television program 130. Other fields are described in 
greater detail below. Any number of records R5 may be 
stored in this database. 

5 

FIG. 5 is a pictorial representation of the layout and 
the information stored in the character database 500. 
The character database 500 comprises information related 
to specific characters for which supplemental audio 

10 information is available. The character database 500 
includes a character name field 502, a character number 
field 504, and a program identification number field 506 
(corresponding to field 404 in program database 400) . 
While only four records R6-R9 for four characters are 

15 depicted in FIG. 5, any number of records may be stored. 

FIG. 6 is a pictorial representation of the layout of the 
recording database 600. The recording database 600 
comprises information related to the supplemental audio 

20 information which may be provided to a particular 

customer 101. The recording database 600 includes a 
program identification number field 602 (corresponding to 
field 404 in the program database 400) , an episode number 
field 604, a character number field 606 (corresponding to 

25 field 504 in the character database 500), a time-code 
received field 608, a recording number field 610, and a 
telephone number dialed field 612. The time-code 
received field 608 is discussed in more detail below. 
Any number of records R10 may be stored in this database. 

30 

FIG. 7 is a pictorial representation of the layout and 
the information stored in the billing database 700. The 
billing database 700 includes a caller telephone number 
field 702, a 900 number dialed field 704, a program 
35 identification number field 706 (corresponding to field 
404 in the program database 400) , a character number 
field 708 (corresponding to character number field 504 in 
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the character database 500) , a recording number field 710 
{corresponding to recording number field 610 in the 
recording database 600), a total time expired field 712, 
a final dollar amount 714, and a caller verified rate 
5 field 716. While only four records R11-R14 are depicted 
in FIG. 7, any number of records may be stored. To 
conserve storage space on the data storage device 226, 
records may be transferred to a different storage device 
or deleted after some predetermined time interval or 
10 after a predetermined event such as receipt of payment 
from the customer 101. 

It is understood that the records and information stored 
within all the databases may be updated as needed, for 
15 example, when a new character is added or a show is 

deleted from the databases. This is accomplished through 
a maintenance terminal 240 operatively connected to the 
service controller 200. 

.20 FIG. 8 is a flow chart depicting an exemplary operation 

of the preferred embodiment of the present invention from 
the point of view of a customer 101. The process starts 
when the customer 101 sees the service information 131 
displayed on a TV screen while viewing the television 
25 program 130 in step 802. 

As mentioned above, the service information 131 comprises 
three components: a 900 telephone number, selection 
information and synchronization information, as described 
30 in detail below. In this embodiment, one general 900 

telephone number is provided for all the TV programs, and 
the selection information, is used to select particular 
supplemental information for the television program 130. 

35 The synchronization information comprises a running time- 
code that is updated on the TV screen at predetermined 
intervals. The preferred interval for updating the time- 
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code is every second. However, other intervals can also 
be used, for example, the time-code could be updated at 
specific milestones during the television program 130. 

5 The time -code ensures that the supplemental audio 

information is properly synchronized to the TV program 
130. It also allows the supplemental audio information 
to be synchronized to a TV program that has been taped 
using a video cassette recorder (VCR) . Thus, the 
10 customer 101 may listen to supplemental audio information 
for a TV program while it is being broadcast, or during a 
time-shifted playback of a previously broadcast TV 
program . 

15 An example of the time- code in the preferred embodiment 
is the number "3600." In this case, the time-code 
indicates that the supplemental audio information should 
be offset by sixty minutes and zero seconds (i.e., 3600 
seconds, which is exactly sixty minutes) . Of course, 

20 time -codes greater than four digits can also be used if 
greater offsets are required. 

As understood by one of ordinary skill in the art of 
telephony, 900 telephone numbers provide services (e.g. 
25 weather or sports information) for which the caller is 
charged a pre-determined rate per minute of use. In a 
similar manner, the customer 101 is charged for 
supplemental audio information provided. 

30 In an alternative embodiment, a unique 900 telephone 
number for each television -program 130 may be used, 
instead of one general 900 telephone number. In such an 
embodiment, the service controller 200 determines which 
television program 130 is being viewed by the customer 

35 101 based on the unique 900 telephone number dialed. In 
another embodiment, .800 or 888 numbers may also be used. 
In such an embodiment, the customer 101 is prompted for a 

25 
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credit card number to pay for the supplemental audio 
information. Arrangements could also be made for pre- 
paid minutes or service credits for particular customers. 

5 By utilizing his or her telephone, the customer 101 
communicates to the service controller 200 by making a 
conventional telephone call. In step 804, the customer 
101 dials the 900 telephone number, which is provided as 
part of the service information 131, and is connected to 

10 the service controller 200. After the connection is 
made, the customer 101 is prompted via recorded voice 
messages to enter selection information and 
synchronization information related to the program being 
viewed. The telephone's keypad is used to enter this 

,15 information. Alternatively, voice recognition software 
within the IVRU 112 could also be used to respond to 
voice commands from the customer 101. 

After entering the synchronization information consisting 
20 of the time-code, the customer 101 can optionally adjust 
the entered time-code to ensure a close match with the 
running time-code on the television program 130. In 
particular, recorded messages transmitted from the IVRU 
112 can prompt the customer 101 to adjust the time-code 
25 using the telephone keypad. For example, the "pound" key 
can be used to increment the value of the time-code, and 
the "star" key to decrement the value of the time-code. 

The customer 101 then listens to a menu listing of 
30 characters. In step 806, the customer 101 selects which 
character (or group of characters) in the television 
program 130 for which he or she desires to receive 
supplemental audio information. Optionally, the customer 
101 may be provided with additional menu choices such as 
35 the option of purchasing supplemental audio information 
for the entire television program 130 or just a portion. 
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In step 808, the service controller 200 plays the 
selected supplemental audio information synchronized to 
the action occurring on the television program 130. The 
supplemental audio information is synchronized so that it 
5 does not overlap with the spoken dialogue of the 
television program 13 0, but supplements it during 
appropriate pauses or gaps. The customer 101 receives 
the supplemental audio information via the telephone. Of 
course, the customer 101 can use a speaker phone or other 
10 speaker means to facilitate listening to the supplemental 
audio information. 

In an alternative embodiment, an interface via the 
Internet may also be provided allowing the customer 101 

15 to receive the supplemental audio information, as well as 
make the request for the supplemental audio information. 
This Internet interface may also be used to receive 
supplemental video or text information from the service 
, controller 200 related to the particular television 

20 program 130 selected by the customer 101. For example, 
the text of a note handed between two characters on a TV 
program 130 could be displayed on a commercially 
available web browser running on a computer connected to 
the Internet. 

25 f 

The service charges are billed to the customer's periodic 
telephone statement in step 810. In the customer's 
periodic telephone statement, an itemized listing of the 
charges appear for each supplemental audio information 

30 received. Of course, the customer 101 may be billed in 
other ways, such as a separate statement for each use. 
Alternatively, instead of a 900 telephone number, an 800 
or 888 telephone number may be provided through which the 
customer 101 would also be prompted to enter a valid 

35 credit card number to be billed for the service. 
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FIGS. 9A and 9B are flow charts of the exemplary 
operation of the service controller 200 after the 
customer 101 dials the 900 telephone number and is 
connected to the service controller 200 via the PSTN 110. 
5 The steps of the process shown in FIGS. 9A and 9B may be 
implemented in a computer program (not shown) that may be 
installed at the service controller 200 from a computer 
readable medium and then stored therein in one or more of 
the ROM 222, the RAM 223 and the data storage device 226 
10 (shown in FIG. 2) . 

After the incoming call has been received by the service 
controller 200 in step 902, the CPU 221 creates in step 
904 a new record in the billing database 700 (shown in 

15 FIG. 2) . Stored in the appropriate fields (702 and 704, 
respectively in FIG. 7) in this new record are the 
caller's telephone number and the 900 telephone number 
dialed by the caller. As understood by one of ordinary 
. skill in the art of telephony, the caller's telephone 

20 number is received from a telephone switch that 
established the connection through the PSTN 110. 

In steps 906-908, the CPU 221 requests and receives the 
program identification code from the caller via the IVRU 

25 112. The program identification code is processed, as 
discussed in detail below, by the service controller 200 
in step 912. Also in step 912, the CPU 221 accesses the 
program database 400 and retrieves the list of 
participating characters stored in participating 

30 characters field 410 for the particular television 

program 13 0. The list is then transmitted to the IVRU 
112. 

In step 914, the caller (i.e. the customer 101) receives 
35 a recorded voice message from the IVRU 112 comprising a 
menu of the participating characters to choose from. The 
caller's character selection is captured and transmitted 
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to the CPU 221 by the IVRU 112 in step 916. This 
information is processed by the service controller 200 
which then accesses the recording database 600 . 

5 The recording database 600 contains identifying 

information for all the supplemental audio information 
stored within the data storage device 226 or accessible 
by the CPU 221. Based on the program identification code 
and character selection entered by the caller, the CPU 

10 221 in step 920 obtains the appropriate supplemental 

audio information stored in the data storage device 226 
or accessible by the CPU 221. 

Before the supplemental audio information is 
15 communicated, the caller is prompted to confirm the 

transaction in step 922. This includes confirming the 
supplemental audio information selected by the caller as 
well as billing information. If the caller decides not 
to complete the transaction, he or she can simply hang up 
20 the telephone or start the process over to select 
different supplemental audio information. 

After confirmation, the CPU 221 in step 924 communicates 
• the supplemental audio information, which is synchronized 

25 to the action occurring on the television program 130 
using the time-code entered by the caller. This is 
accomplished, as described above, by starting the 
supplemental information using the time -code as an 
offset. The CPU 221 starts playback of the supplemental 

30 audio information accordingly, based on the entered time- 
code . 

After the call termination (i.e. the supplemental audio 
information is complete or the caller hangs up the 
35 telephone), the CPU 221 in step 926 updates the caller's 
record in the billing database 700. This record is used 
to track the cost of the call. The actual billing is 
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done by the phone company, who, upon receipt of payment 
from the caller, reimburses the service for the cost of 
the content. The service charge information is sent to 
the caller's telephone service provider and appears on 
5 the caller's monthly or periodic telephone statement. Of 
course/ alternate methods of payment may be used instead, 
including credit card transactions, debit cards, or the 
caller could receive a separate bill for each use and pay 
by check or similar means. In these cases, the billing 
10 database would be used to actually bill the customer 101 
and/or interact with a credit card company. 

The revenues generated by the service can be distributed 
in a variety of ways among the various persons and 
15 organizations providing the service (i.e. the TV program 
provider, the 900 service provider, the actors, etc.). 

FIC5S. 10A and 10B are flow charts depicting how the 
service controller processes information received from 
20 the IVRU 112. In step 1002, the CPU 221 receives the ten 
digit alphanumeric string from the IVRU 112 . These ten 
digits represent the program identification code which is 
entered by the customer 101 via his or her telephone, as 
discussed above. 

25 

In step 1004, the CPU 221 separates the ten digit string 
as follows: the first two digits, the next four digits, 
and the last four digits. As described below, the 
separated strings represent program selection information 
30 and synchronization information related to the broadcast 
television program 130. 

The first two digits represent the network identification 
number for the television program 130. In step 1006, the 
35 CPU 221 searches for a match in the network 

identification number field 304 of the network database 
300. If no match is found in step 1008, the CPU 221 
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prompts the customer 101 via the IVRU 112 to reenter the 
program identification code in step 1010. Otherwise, the 
CPU 221 continues to step 1012. 

5 The next four digits of the program identification code 
represent the program identification number. In step 
1012, the CPU 221 searches for a match in the program 
identification number field 404 of the program database 
400. If no match is found in step 1014, the CPU 221 
10 prompts the customer 101 via the IVRU 112 to reenter the 
program identification code in step 1016. Otherwise, the 
CPU 221 continues to step 1018. 

Alternatively, the program identification code can have 
15 more than ten digits. This would allow additional 

information to be entered by the caller, for example, an 
episode number. The last four digits of the program 
identification code represent the synchronization 
information (i.e. the time - code > . 

20 

In step 1018, the CPU 221 stores the time-code in a new 
record created for the customer 101 in the recording 
database 600. Once the time-code has been received (and 
adjusted) , . the CPU 221 updates the time-code at the 
25 predetermined intervals to ensure proper synchronization 
with the television program 130 is maintained. Other 
information including the program identification number 
and the number dialed by the caller are also stored in 
the new record. 

30 

In an alternative embodiment, the synchronization 
information may be derived by the service controller 200 
directly from the television program 130. The user, for 
example, could hold the telephone up to the speaker of 
35 the television so that the service controller 200 could 
receive the synchronization information. The service 
controller 200 would monitor the audio component of the 
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television program 130 and compare it with the 
corresponding audio track (i.e. a duplicate) stored at 
the service controller 200. Using the stored copy, the 
service controller 200 would synchronize the supplemental 
5 audio information without requiring the customer 101 to 
enter any additional information. 

The CPU 221 accesses the participating characters field 
410 in the program database 400 and transmits the list to 

10 the caller via the IVRU 112 in step 1020. The CPU 221 
receives from the caller via the IVRU 112 the character 
selection number in step 1022. Once again, the caller 
enters this information using his or her telephone 
keypad. The character selection number is a two digit 

15 alphanumeric string. The CPU 221 receives it and enters 
it in the appropriate field in the recording database 600 
in step 1024 . 

In an alternative embodiment, the supplemental audio 
20 information may be used to offer services such as foreign 
language translations of TV programs and descriptive 
audio for the blind. 

FIG. 11 is a schematic block diagram overview of one 
25 system constructed in accordance with the present 

invention. A television receiver 1120 receives a video 
program from a video source 1110 while a customer 101 
observes the video program. While one customer is 
depicted receiving the video program from the television 
30 receiver 1120, it is understood that the video program 
may be broadcast to and/or viewed by multiple customers. 

As depicted in FIG. 11, the video program is received and 
displayed by the television receiver 1120 located in 
35 proximity to the customer 101. Of course, other types of 
video signal display devices may be used instead of a 
conventional television set, such as, a personal computer 
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adapted to receive and display video signals. The 
television receiver 1120 may receive the video program 
through an antenna adapted to receive TV signals from a 
TV broadcast station, from a satellite transponder, a 
5 trunk cable from a CATV (i.e. cable TV) system, or from 
any other suitable transmission means, as well as a 
playback of a pre-recorded program from a video cassette, 
laserdisc or compact disc. 

10 As illustrated in FIG. 12, program identification 

information 1126 and synchronization information 1124 are 
displayed discretely on a display screen 1122 of the 
television receiver 1120. This information is integrated 
with the video program. The location of this information 

15 may be anywhere on the display screen 1122 . The amount 
. of display area consumed, by the information (i.e., font 
size) may also be varied. Program identification 
information 1126 and synchronization information 1124 may 
also be presented in a non- visual machine readable format 

20 such as by embedding data in an audio track or using a 
portion of a video track not displayed on the television 
receiver 1120. 

The program identification information 1126 and the 
25 synchronization information are recorded on the same 
recording medium as the video program. For example, a 
television program can be recorded and then the 
synchronization information can be added during editing 
of the recorded television program. Alternatively, this 
30 information can be added at the time of recording or 
broadcasting the television program. It is also 
understood that the program identification information 
1126 and the synchronization information 1124 would be 
presented even if the video program is recorded on a 
35 recorded medium and played-back by the customer 101. 
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As shown in FIG, 12, the program identification 
information 1126 is an eight digit alphanumeric code 
uniquely representing the particular video program being 
displayed. It is understood that other types of 
5 identification tags may be used to distinguish video 
programs from each other such as graphical images or 
other symbols. Each video program contains one such code 
so that customers can request specific supplemental 
information related to that video program, as discussed 
10 below. 

The synchronization information 1124 comprises a running 
time-code that is updated on the display screen 1122 at 
predetermined intervals. The preferred interval for 

15 updating the time-code is every second. However, other 
intervals can also be used, for example, the time-code 
could be updated at specific milestones during the video 
program such as after each commercial break or every few 
minutes . 

20 ! . 

As discussed above, the time-code ensures that the 
supplemental information is properly synchronized to the 
video program. It also allows the supplemental 
information to be synchronized to the video program that 

25 has been taped using a video cassette recorder ("VCR') . 
Therefore, the viewer may receive supplemental 
information for a TV program while it is being broadcast, 
or during a time-shifted playback of a previously 
broadcast TV program. 

30 

An example of the time-code in the preferred embodiment 
is "00:05:36." In this example, the time-code indicates 
that the supplemental information should be offset by 
five minutes and thirty-six seconds from the start of the 
35 video program. Of course, other time-codes formats can 
be used as well . 
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Referring back to FIG 11., the customer 101, by utilizing 
a personal computer 1140, communicates over a computer 
data communication network 1150 to a web site server 
1160. The personal computer 1140 includes a CPU 1142, 
5 and a storage device 1144. The web site server 1160 
includes a CPU 1162, a receiver/transmitter 1166, and a 
storage device 1164. The web site server 1160 provides 
for storage of the supplemental information related to 
the various video programs on a server .connected to the 

10 Internet. The supplemental information is stored in 
units, referred to as "web pages," which may include 
textual, graphic, audio and/or video information and 
hyperlinks to other information related to the particular 
video program. Specific portions of the web pages such 

15 as hyperlinks may be activated in synchronization with 
certain events during the video program. 

Viewer 1130 can access the web site server 1160 through 
conventional interface software called a "web browser" or 

20 other data communication protocol means. To gain access 
to the web site server 1160, for example, customer 101 
specifies the Internet address or URL associated with the 
web page to the web browser. The browser then 
communicates with the relevant web site server 1160 

25 containing the web page data. A modem (not shown) or 
similar means can be used to facilitate communication 
with the web site server 1160. In the present invention, 
the URL of the web site server 1160 can be displayed 
before and/or during the video program. The URL can also 

30 be generally distributed in advertisements. For example, 
the URL can be printed in newspaper and magazine 
advertisements or displayed during a televised 
advertisement. 

35 After accessing the web site server 1160, customer 101 is 
guided by a graphical user interface and icons (not 
shown) to access the supplemental information stored 

35 



CA 02283940 1999-09-13 



WO 98/43406 PCT/US98/05112 

therein. In particular, customer 101 provides the 
program identification information 1126 and 
synchronization information 1124, to the web site server 
1160. In response, the web site server 1160 first 
5 accesses the particular supplemental information related 
to the program identification information 1126 provided 
by customer 101. Then the web site server 1160 
synchronizes the supplemental information to the action 
and events occurring in the video program based on the 
10 synchronization information 1124 (i.e., the time-code) . 
This is accomplished by using the time-code as an offset, 
as described above. The supplemental information, in the 
form of web pages, is then sent to customer 101 via the 
personal computer 1140. 

15 

In one embodiment, customer 101 is prompted for a credit 
card number to pay for the supplemental information when 
accessing the web site server 1160. Arrangements could 
also be made for periodic service bills to be mailed to 

20 the customer 101. Alternative payment mechanisms include" 
billing in conjunction with a 900, 800 or 888 telephone 
number, Digicash or Cybercash. The revenues generated by 
the service can be distributed in a variety of ways among 
the various persons and organizations providing the 

25 service (i.e. the video program provider, the Internet 
service provider, the actors, etc . ) . 

When customer 101 receives the web pages, the information 
contained therein preferably corresponds to the action 

30 and events occurring in the video program. This ensures 
that the events contained within the supplemental 
information coincides with the content of the video 
program. Each event has a respective begin time and end 
time which is synchronized to the time -code, as shown in 

35 the example below: 
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BEGIN-TIME 


END -TIME 


EVENT 


00:00:00 


00:02:00 


display intro. images 


00:02 : 01 


00:03:36 


show close-up of report 


00:03:37 


00:10:56 


play camera angle #1 


00:10:57 


00:12:00 


show product menu 


00:12:01 


00:20:00 


play scene #1 


00:20:01 


00:21:08 


show close-up of note 


00:21:09 


00 :26 :11 


play audio #1 


00:26 :12 


00:30:00 


display closing images 



10 

Some of the supplemental information contained in the web 
site server 1160 may be selectable by customer 101. For 
example, customer 101 may choose to receive one or more 
close-ups of several documents, medical reports, suicide 

15 notes, etc. These choices are made available at the 
appropriate times throughout the video program. In 
addition, customer 101 may wish to select a particular 
character within a TV show and receive supplemental 
scenes and dialogue related to this character. Once 

20 again, these scenes are synchronized to accommodate when 
that character is "in" or "out" of the scene being 
displayed by the television receiver 1120. 

FIG. 13 is a schematic block diagram illustrating a 
25 preferred integrated display device 1300 of the present 
invention. This device integrates the functions of the 
television receiver 1120 and the personal computer 1140 
discussed above. The integrated display device 1300 
allows the video program and the supplemental information 
30 to be viewed on a single display 1313. Any one or a 
variety of display apparatuses are suitable for this 
purpose, including, for example a Cathode Ray Tube 
("CRT"), Liquid Crystal Display ("LCD"), or similar 
apparatus capable of displaying high resolution color 
35 video signals. The display 1313 has segregated sections 
to display the video program, the supplemental 
information and a control/menu section (not shown) . An 
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input/output interface 1308 transmits the outgoing 
video/control signal to the display 1313. . 

The integrated display device 1300 includes a CPU 1301 
5 which performs the processing functions. The integrated 
display device 1300 also includes a read only memory 1302 
(ROM) and a random access memory 1303 (RAM) . The ROM 
1302 is used to store at least some of the program 
instructions that are to be executed by the CPU 1301, 
10 such as portions of the operating system or basic input - 
output system (BIOS) , and the RAM 1303 is used for 
temporary storage of data. A clock 1304 provides timing 
signals which are required by the CPU 1301. 

15 The CPU 1301 can also store information to, and read 
information from, a data storage device 1306. This 
storage device preferably includes or is adapted to 
receive static memory capable of storing large volumes of 
data, such as one or more of floppy disks, hard disks, 

20 CDs, or magnetic tapes. 

The data storage device 1306 includes a simulcast event 
database 1500. This database is described more fully 
with reference to FIG. 15, below. in addition, the data 

25 storage device 1306 includes instructions (not shown) 
which can be read by and executed by the CPU 1301, 
thereby enabling the CPU 1301 to process requests for 
supplemental information and control the data flow 
therein. Additional databases may be added as needed to 

30 store a variety of other information that may be required 
for other purposes . 

The integrated display device 1300 also includes a 
communication port 1305 connected to a computer data 
35 communication network interface 1310. The communication 
port 1305 enables the CPU 1301 to communicate with 
external devices, in particular, with the web site server 
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1160 discussed above. The network interface 1310 serves 
as an interface between the CPU 1301, via the 
communication port 1305, and the web site server 1160. 
Network interface 1310 is capable of receiving data 
5 streams from both the CPU 1301 and the web site server 
1160, and the data streams are formatted according to 
their respective communication protocols. Network 
interface 1310 can also reformat the data streams 
appropriately and relay the data streams accordingly. 
Network interface 1310 preferably can handle a variety of 
different communication protocols. 

A video processor 1307 is used by the integrated display 
device 1300 to receive video signals from a video input 
source 1311. The video signals can be in either an 
analog or digital format. These signals are converted 
into a digital format suitable for storage and 
manipulation by the CPU 1301. 

20 An input device 1312 is also connected to the 

input/output interface 1308. The input device 1312 is 
used to facilitate input of data by the customer 101 from 
either a mouse or keyboard. However, other input devices 
would be suitable for this purpose as well, including, 

25 for example, joysticks, trackballs, infra-red hand-held 
remote control units , video cameras, or even speech 
recognition devices. The input device 1312 may interface 
to CPU 1301 via input/output interface 1308, as shown in 
FIG. 13. Alternatively, input device 1312 may directly 

30 interface with the CPU 1301. The input device 1312 may 
also comprise a floppy disk drive, CD-ROM drive, or other 
such devices which are capable of reading data from 
computer readable media. 



35 



FIG. 15 shows an example; of the simulcast event database 
1500 within the integrated display device 1300. The 
simulcast event database 1500 comprises information 
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related to the supplemental information available for 
each video program. This database includes a series ID 
field 1502, an episode ID field 1504, a begin time code 
field 1506, an end time code field 1508, and an event 
5 instruction field 1510. The series ID field 1502 
corresponds to the first four digits of the program 
identification information 1126 associated with each 
video program, while the episode ID field 1504 
corresponds to the last four digits. The begin and end 

10 time code fields 1506 and 1508, respectively, indicate 
when certain events related to the supplemental 
information are to occur as discussed above. These 
events are synchronized to the video program using the 
synchronization information 1124 (i.e., the time-code) . 

15 The event instruction field 1510 indicates the type of 

events (i.e., supplemental information) that the customer 
101 will receive at the specified times within the video 
program. 

20 While only two records are depicted in FIG. 15, any 
number of records may be stored. To conserve storage 
space on the data storage device 13 06, records may be 
transferred to a different storage device or deleted 
after some predetermined time interval or after a 

25 predetermined event (e.g., after the video program has 
been viewed) . 

The information stored in the simulcast event database 
1500 can be updated in a number of ways. The integrated 

30 display device 1300 may communicate with the web site 

server 1160 and download the information related to a one 
or more video programs. This may be done before or 
during the viewing of the video program. It is 
understood that the information stored within the 

35 simulcast event database . 1500 may be updated as needed. 
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In an alternative embodiment, the supplemental 
information can be downloaded into the simulcast event 
database 1500 from a computer readable medium, such as a 
CD-ROM, floppy disk, and the like. In this embodiment, 
5 the integrated display device 1300 would execute a stand- 
alone computer application designed to synchronize and 
display the supplemental information. The computer 
application could be distributed, for example, via a CD- 
ROM or the Internet. 

10 

The personal computer 1140 could also execute a stand- 
alone computer application coupled with an integrated 
simulcast event database 1500 to provide synchronized 
supplemental information to the customer 101. In this 

15 embodiment, the customer 101 would not be required to 
access the web site server 1160 each time supplemental 
information related to a particular video program was 
desired. The customer 101 would simply download the 
computer application and the simulcast event database 

20 1500 beforehand and execute the computer application when 
viewing the video program. Alternatively, as discussed 
above, the computer application and database could be 
distributed using other means such as a CD-ROM. 

25 FIG. 14 is a flow chart of the exemplary operation of the 
stand-alone computer application for presenting the 
supplemental information. The process will be described 
with reference to integrated display device 1300 , however, 
the method could also be implemented using personal 

30 computer 1140. The process starts when the customer 101 
receives the program identification information 1126 and 
the synchronization information 1124 displayed on display 
screen 1122 while viewing the video program in step 1402. 
In step 1404, the computer application is initialized by 

35 the customer 101. This can be performed, for example, 
using a mouse to double click an icon representing the 
computer application. The customer 101 is then prompted 
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to enter the program identification information 1126 and 
the synchronization information 1124 via the input device 
1312 in step 1406. 

5 After entering the synchronization information 1124 

consisting of the time -code, customer 101 can optionally 
adjust the entered time-code to ensure a close match with 
the running time -code contained within the video program. 
In particular, pop-up messages and menu selections prompt 
10 and guide the customer 101 to adjust the time-code using 
the input device 1312. This can also be accomplished 
using the keyboard or mouse (not shown) of the personal 
computer 1140. 

15 In step 1408 , the program identification information 1126 
is processed. Based on the program identification 
information 1126 entered by customer 101, the CPU 1301 
obtains the appropriate supplemental information stored 
in the simulcast event database 1500 or accessible by the 

20 CPU 1301 via external means (i.e./ from a CD-ROM or the 
web site server 1160). The CPU 1301 also initializes a 
. time counter based on the synchronization information and 
updates the time counter based on timing signals from 
internal clock. The time-code entered by customer 101 is 

25 incremented (step 1410) at predetermined intervals (e.g., 
every second) to ensure that proper synchronization is 
maintained with the video program. 

The CPU 1301 determines in step 1412 when to present 
30 specific events (part of the supplemental information) 
based on the continually updated time counter. For 
example, when the updated time counter is greater-than or 
equal to the BEGIN-TIME but less -than the END-TIME for a 
specific event, that event is presented to customer 101. 
35 The first event presented to the customer 101 is thereby 
synchronized to the video program. The first event is 
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off-set using the time-code information, so that proper 
synchronization with the video program is maintained. 

In step 1414, the integrated display device 1300 displays 
5 the event synchronized to the action occurring on the 
video program. 

As discussed above; customer 101 may also select various 
options while the supplemental information is being 

10 presented. When optional or selectable information is 

available, customer 101 is prompted by various means such 
as: pop windows, menu choices, dialog boxes, etc. 
appearing on the display 1313, Methods of implementing 
these applications will be apparent to those skilled in 

15 the art, and will not be further described in the present 
disclosure. Customer 101 can then exercise an option via 
an input device such as input device 1312. In step 1416, 
the response of customer 101 is processed and the 
appropriate supplemental information is presented 

20 accordingly. 

The process then loops back to step 1410 and is repeated 
until the end of the video program or customer 101 
terminates the computer application. 

25 

In an alternative embodiment, the program identification 
information 1126 and the synchronization information 1124 
may be derived by the integrated display device 13 00 
directly from che video program. The video processor 
30 1307 of the integrated display device 1300 extracts the 
necessary information as needed. Since this process 
would be transparent to the customer 101, the information 
would not need to be visually displayed. 

35 While the present invention has been described above in 
terms of specific embodiments, it is to be understood 
that the invention is not intended to be confined or 
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limited to the embodiments disclosed herein. On the 
contrary, the present invention is intended to cover 
various methods, structures and modifications thereof 
included within the spirit and scope of the appended 
5 claims. 
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WHAT IS CLAIMED IS: 

11. A data processing apparatus for providing 

2 supplemental broadcast information, comprising: 

3 a CPU; 

4 a storage device operatively connected to said CPU; 

5 an apparatus, adapted for communicating with said 

6 CPU, for receiving a request from a caller over a 

7 telephone network for supplemental information related to 

8 a broadcast television program and for relaying the 

9 request to said CPU; and 

10 said storage device storing a program, adapted to 

11 be executed by said CPU, for processing the request for 

12 supplemental information and for transmitting the 

13 requested supplemental information through said 

14 apparatus . 

1 2. The apparatus according to Claim 1, wherein the 

2 supplemental information comprises audio information 

3 recorded on a recording medium. 

1 3. The apparatus according to Claim 2, wherein said 

2 information recorded on the recording medium comprises 

3 supplemental dialogue for a character within the 

4 broadcast television program. 

1 4. The apparatus according to Claim 3, wherein a 

2 playback of said supplemental dialogue does not overlap 

3 dialogue spoken by any character within the broadcast 

4 television program. 

1 5. . The apparatus according to Claim 2, wherein said 

2 storage device contains a database to store the 

3 supplemental information, and said program stored within 

4 said storage device is further adapted to perform 

5 maintenance related to said database. 
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1 6. The apparatus according to Claim 1, wherein the 

2 supplemental information comprises live audio 

3 information. 

1 7. The apparatus according to Claim 1, wherein said 

2 apparatus comprises an IVRU including a plurality of 

3 voice messages, and wherein said voice messages are used 

4 to communicate to, and obtain information from, the 

5 caller. 

1 8. The apparatus according to Claim 1, wherein the 

2 request for supplemental information includes providing 

3 selection information and synchronization information. 

1 9. The apparatus according to Claim 8, wherein the 

2 synchronization information comprises a time-code; and 

3 said program is further adapted to update said time- 

4 code at predetermined intervals, and to use said time- 

5 code to synchronize the supplemental information to an 

6 audio component of the broadcast television program. 

1 10. A method for providing supplemental information 

2 synchronized to a broadcast television program using a 

3 CPU, and a storage device operatively connected to the 

4 CPU and containing a program adapted to be executed by 

5 the CPU for processing a request for supplemental 

6 information, and an apparatus adapted for communicating 

7 with the CPU, said method comprising the steps of: 

8 receiving the request for supplemental information 

9 from a caller via the apparatus; 

10 receiving selection information from the caller via 

11 the apparatus; 

12 processing the selection information by having the 

13 CPU execute the program; 

14 receiving synchronization information ; 
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15 synchronizing the requested information with to an 

16 audio component of the broadcast television program using 

17 the synchronization information; 

18 communicating the requested information via the 

19 apparatus. 

1 11, The method according to Claim 10, further comprising 

2 the step of updating a database within the storage device 

3 indicating that the requested supplemental information 

4 has been communicated in said communicating step. 

1 12. The method according to Claim 10 , further comprising 

2 the step of requesting payment for the communicated 

3 supplemental information. 

1 13. A method for providing entertainment, comprising: 

2 receiving a request from a requestor for 

3 supplemental' information relating to a television program 

4 being broadcast at the time of the request; 

5 processing the request for supplemental information; 

6 transmitting the requested supplemental information 

7 to the requestor at times when an audio component of the 

8 television program and the transmission of the requested 

9 supplemental information do not conflict with each other. 

1 14. Computer executable process steps, stored on a 

2 computer readable medium, for processing requests for 

3 supplemental information related to a broadcast 

4 television program, comprising; 

5 * step to receive a supplemental information request 

6 via a telephone network interface; 

7 a step to receive selection information; 

8 a step to process the selection information; 

9 a step to retrieve the requested supplemental 
10 information; 
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11 a step to synchronize the requested supplemental 

12 information to an audio component of the broadcast 

13 television program; and 

14 a step to output the synchronized requested 

15 supplemental information. 

1 15, The computer executable process according to Claim 

2 14, further comprising the step of receiving 

3 synchronization information related to the broadcast 

4 television program. 

1 16. A system, comprising: 

2 means for broadcasting a television program 

3 including an audio component and a video component; 

4 at least one supplemental component for said 

5 broadcast television program; 

6 a data processing apparatus including a CPU and a 

7 storage device operatively connected to said CPU, said 

8 storage device also containing a program, adapted to be 

9 executed by said CPU, for receiving a request for said 

10 supplemental component , processing the request for said 

11 supplemental component, and providing a playback of the 

12 requested supplemental component which is time 

13 synchronized to said audio component of said broadcast 

14 television program. 

1 17. The apparatus according to Claim 16, wherein the 

2 broadcast television program is selected from the group 

3 consisting of a live broadcast and a prerecorded 

4 broadcast . 

1 18. The apparatus according to Claim 16, wherein the 

2 playback of the requested supplemental information is 

3 transmitted via a telephone network interface. 

1 19. The apparatus according to Claim 16, wherein said 

2 video component of said broadcast television program 
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3 contains synchronization information and selection 

4 information. 

1 20. The apparatus according to Claim 19, wherein the 

2 program is further adapted to receive and process said 

3 synchronization information. 

1 21. The apparatus according to Claim 16 , wherein the 

2 program is further adapted to receive the request for 

3 said supplemental component from a telephone network 

4 interface. 

1 22 . A method for providing entertainment using a data 

2 processing apparatus including a CPU and a storage device 

3 operatively connected to the CPU and containing a program 

4 adapted to be executed by the CPU for processing a 

5 request for supplemental information and providing a 

6 playback of the requested supplemental information, said 

7 method comprising the steps of: 

8 .broadcasting a television program including an audio 

9 component and a video component; 

10 receiving a request for the supplemental information 

11 related to the television program, where the request 

12 includes synchronization information; : 

13 processing the request for the supplemental 

14 information by having the CPU in the data processing 

15 apparatus execute the program; 

16 synchronizing the requested supplemental information 

17 to the audio component of the television program using 

18 the synchronization information; and 

19 transmitting the requested supplemental, information. 

1 23. The method according to Claim 22, wherein the 

2 requested supplemental information is live audio 

3 information. 
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1 24. The method according to Claim 22, wherein the 

2 requested supplemental information is prerecorded on a 

3 recording medium. 

1 25. The method according to Claim 22, further comprising 

2 the step of recording on a recording medium the 

3 supplemental information. 

1 26. A method for providing visual and audio 

2 entertainment, comprising the steps of: 

3 broadcasting a television program including an audio 

4 component and a video component; 

5 receiving a request from a requestor for 

6 supplemental information related to the television 

7 program; 

8 processing the request for the supplemental 

9 information; 

10 transmitting the requested supplemental information 

11 to the requestor at times when the audio component of the 

12 television program and the playback of the requested 

13 supplemental information do not conflict. 

1 27. The method according to Claim 26, further comprising 

2 the step of updating a database indicating that the 

3 requested supplemental information has been transmitted 

4 in said transmitted step. 

1 28. The method according to Claim 26, wherein the 

2 broadcasting step further comprises broadcasting ordering 

3 information for the supplemental information. 

1 29. The method according to Claim 26, wherein the 

2 broadcasting step further comprises broadcasting 

3 synchronization information related to the television 

4 program. 
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1 30. A method according to Claim 29, wherein the step of 

2 broadcasting the synchronization information further 

3 comprises broadcasting a time-code that is updated at 

4 predetermined intervals and which is used to synchronize 

5 the supplemental information to the television program 

6 broadcast in said transmitting step. 



10 
11 
12 



31. The method according to Claim 29, wherein the step 
of receiving the request for the supplemental information 
includes receiving the synchronization information from 

t hp rpmipot-nr 



1 32. A method of receiving supplemental information 

2 related to a broadcast television program including an 

3 audio component and an video component, comprising the 

4 steps of.: 

5 viewing the broadcast television program; 

6 receiving ordering information for the supplemental 

7 information from the broadcast television program; 

8 requesting the supplemental information in 

9 accordance with the ordering information; 
providing selection information ; 
receiving the selected supplemental information 

during the broadcast television program. 



1 33. A method according to Claim 32, wherein the step of 

2 receiving ordering information comprises receiving 

3 program identification information, a telephone number 

4 used for ordering the supplemental information, and 

5 synchronization information related to the broadcast 

6 television program. 

1 34. A method according to Claim 32, further including 

2 the step of paying for the supplemental information. 
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1 35. A method according to Claim 34, wherein the step of 

2 paying for the supplemental information is accomplished 

3 by a credit card transaction. 

1 36. A method according to Claim 34, wherein said step of 

2 paying for the supplemental information is performed 

3 through a telephone bill. 

1 37. A method for requesting supplemental information 

2 related to at least one character within a broadcast 

3 television program including an audio component and an 

4 video component, comprising the steps of: 

5 viewing the broadcast television program; 

6 requesting supplemental information for the 

7 character within the broadcast television program; and 

8 receiving the requested supplemental 

9 information for the character synchronized with the audio 
10 component of the broadcast television program. 

1 38. A method for preparing supplemental information 

2 related to at least one character featured on a broadcast 

3 television program, said method comprising the steps of: 

4 writing dialogue, not broadcast with the television 

5 program, for the character featured on the broadcast 

6 telievision program; 

7 recording the additional dialogue on a recording 

8 medium; 

9 editing the additional dialogue recorded in said 

10 recording step such that the audio component of the 

11 broadcast television program and the edited additional 

12 dialogue do not conflict with each other when listened to 

13 simultaneously. 

1 39. A server apparatus for providing supplemental 

2 information, comprising: 
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14 



3 a controller; 

4 a storage device operatively connected to said 

5 controller; 

6 a receiver/transmitter, adapted for 

7 communicating with said controller, for receiving over a 

8 data communication network a request for supplemental 

9 information related to a video program and for receiving 

10 synchronization information related to the video program, 

11 and for relaying the request and the synchronization 

12 information to said controller; and 
said storage device in said controller 

containing a program, adapted to be executed by said 

15 controller, for processing the request for supplemental 

16 information, synchronizing the supplemental information 

17 to the video program, and for transmitting the 

18 synchronized supplemental information through said 

19 receiver/transmitter over the data communication network. 

1 40. The apparatus according to Claim 39, wherein the 

2 supplemental information comprises audio and visual 

3 information recorded on a recording medium. 

1 41. The apparatus according to Claim 40, wherein said 

2 storage device contains a database to store the 

3 supplemental information, and said program within said 

4 storage device is further adapted to perform database 

5 maintenance . 
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1 42. The apparatus according to Claim 39, said program 

2 is further adapted for receiving program identification 

3 information related to the video program and for 

4 transmitting the program identification information 

5 through said receiver/transmitter over the data 

6 communication network, 

1 43. The apparatus according to Claim 39, wherein the 

2 synchronization information comprises a time-code; and 

3 said program is further adapted to update said 

4 time -code at predetermined intervals. 

1 44 . The apparatus according to Claim 39 , wherein said 

2 receiver/transmitter is adapted for receiving additional 

3 requests from a requestor to interactively change the 

4 requested supplemental information based upon the viewed 

5 program; and 

6 said program is further adapted to change and 

7 transmit the requested supplemental information in 

8 accordance with the additional requests from the 

9 requestor. 

1 45. A method for providing supplemental information 

2 using a data processing apparatus including a CPU and a 

3 storage device operative ly connected to the CPU and 

4 containing a program adapted to be executed by the CPU 

5 for processing a request for the supplemental information 
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6 related to a video program and providing the requested 

7 supplemental information, said method comprising the 

8 steps of : 

9 receiving a request for the supplemental 

10 information related to the video program; 

11 receiving synchronization information related 

12 to the video program; 

13 processing the request for the supplemental 

14 information by having the CPU in the data processing 

15 apparatus execute the program; 

16 synchronizing the requested supplemental 

17 information to the video program using the 



18 
19 



synchronization information; and 

transmitting the requested supplemental 



20 information. 

1 46. The method according to Claim. 45, wherein the 

2 requested supplemental information comprises audio and 

3 visual information. 

1 47. The method according to Claim 45, further 

2 comprising the step of providing program identification 

3 information prior to processing the request. 

1 48. The method according to Claim 45, wherein. the 

2 synchronization information comprises a time -code which 

3 is updated at predetermined intervals. 
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1 49. The method according to Claim 45, further 

2 comprising the step of: 

3 receiving additional requests to interactively 

4 change the requested supplemental information based upon 

5 the viewed video program; and 

6 changing the requested supplemental information in 

7 accordance with the additional requests . 

1 50. Computer executable process steps, stored on a 

2 computer readable medium, for processing a request for 

3 supplemental information related to a video program, 

4 comprising : 

5 a step to receive a request for the supplemental 

6 information; 

7 a step to receive program identification information ; 

8 a step to receive synchronization information; 

9 a step to process the program identification 

10 information; 

11 a step to retrieve the requested supplemental 

12 information in accordance with the program identification 

13 information; 

14 a step to synchronize the requested supplemental 

15 information to the video program in accordance with the 

16 synchronization information; and 

17 a step to output the synchronized requested 

18 supplemental information. 
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1 51. The computer executable process according to Claim 

2 50, further comprising a step to change the synchronized 
. 3 requested supplemental information based on additional 

4 instructions . 

1 52. The computer executable process according to Claim 

2 50, further comprising the step of verifying that 

3 synchronization is maintained with the video program. 

1 53 . The computer executable process according to Claim 

2 50, wherein the synchronization information comprises a 

3 time-code which is updated at predetermined intervals. 

I 54. A system, comprising: 
,2 a display device configured to receive a video 

3 program, said video program including an audio component 

4 and a visual component, wherein said video program 

5 includes synchronization information; and 
a data processing apparatus including a CPU and 

7 a storage device operatively connected to said CPU> said 

8 storage device containing a program, adapted to be 

9 executed by said CPU, for transmitting a request for 
supplemental information relating to said video program, 
for transmitting information corresponding to said 
synchronization information, for receiving the 

13 supplemental information, and for displaying the 

14 supplemental information that is synchronized to said 
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15 video program in accordance with said synchronization 

16 information. 

1 55. The system according to Claim 54, wherein said 

2 display device and said data processing apparatus are 

3 integrated. 

1 56, The system according to Claim 54, wherein the 

2 supplemental information is received via a computer data 

3 communication network interface . 

1 57. The system according to Claim 54 , wherein said 

2 visual component of said video program contains said 

3 synchronization inf ormation. 

1 58. The system according to Claim 54, wherein the 

2 program is further adapted to transmit the request for 

3 the supplemental information via a computer network 

4 interface to a server, and where said server transmits, 

5 in response, the; requested supplemental information. 

1 59. The system according to Claim 54, wherein the 

2 program is further adapted to transmit program 

3 identification information and the synchronization 

4 inf ormation via a computer network interface. 
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! 60. The system according to Claim 54, wherein the 

2 program is further adapted to make additional requests to 

3 interactively change the supplemental information. 

1 61. The system according to Claim 54, wherein said 

2 video program is selected from the group consisting of a 

3 live television broadcast and a prerecorded television 

4 broadcast. 

1 62. The system according to Claim 54, where said system 

2 further comprises a reader, adapted to retrieve data from 

3 a computer readable medium, operatively connected to said 

4 data processing apparatus, and where the supplemental 

5 information is received from said reader which retrieves 

6 the supplemental information from said computer readable 

7 medium. 

1 ,63. : The system according to Claim 62, wherein said 

2 reader is a CD-ROM device. 

1 64. A method for receiving supplemental information 

2 related to a video program using a display device for 

3 displaying the video program and a data processing 

4 apparatus including a CPU and a storage device 

5 operatively connected to the CPU and containing a program 

6 adapted to be executed by the CPU for requesting the 
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7 supplemental information and for displaying the 

8 supplemental information, comprising the steps of: 

9 displaying the video program on the display 

10 device, where the video program includes an audio 

11 component, a visual component and synchronization 

12 information; 

13 initiating the program; 

14 requesting the supplemental information; 

15 transmitting information corresponding to the 

16 synchronization information from the video program; 

17 receiving the supplemental information 

18 synchronized to the video program; and 

19 ; displaying, via the data processing apparatus, 

20 . the 'supplemental information. 

1 65. A method for receiving supplemental information, 

2 comprising the steps of: 

3 displaying a video program including an audio 



4 component, a visual component and synchronization 

5 information; 

y - 

6 initiating a computer program adapted to 

7 process a request for supplemental information related to 

8 the action and events occurring within the video program; 

9 requesting the supplemental information; 

10 transmitting information corresponding to the 

11 synchronization information from the video program; 

12 receiving the supplemental information; and 
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13 displaying the requested supplemental 

14 information, synchronized to the video program in 

15 accordance with synchronization information. 

1 66. The method according to Claim 65, wherein the 

2 visual component of the video program contains program 

3 identification information related to the video program. 

1 67. The method according to Claim 65, wherein the 

2 visual component of the video program contains the 

3 synchronization information. 

1 68, A method according to Claim 67, wherein the 

2 synchronization information comprises a time -code that is 

3 changed at predetermined intervals and which is used to 

4 synchronize the supplemental information to the video 

5 program. 

1 69. The method according to Claim 65, further 

2 comprising the step of making additional requests to 

3 change, interactively, the requested supplemental 

4 information. 

1 70. A method of receiving supplemental information 

2 related to a video program that includes an audio 

3 component and a visual component, comprising the steps 

4 of: 
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5 viewing the video program; 

6 receiving program identification information 

7 and synchronization information related to the video 

8 program; 

9 requesting the supplemental information; 

10 providing the synchronization information and program 

11 identification information; 

12 receiving the supplemental information; and 

13 viewing supplemental information while the 

14 video program is in progress. 

1 71. A method according to Claim 70, wherein the step of 

2 receiving synchronization information comprises receiving 
3a time-code which is updated at predetermined intervals. 

1 72. A method according to Claim 70, further including 

2 the step of paying for the supplemental information. 

1 73. A method according to Claim 72, wherein the step of 

2 paying for the supplemental information is accomplished 

3 by a credit card transaction. 

1 74. A method according to Claim 72, wherein said step 

2 of paying for the supplemental information is performed 

3 through payment of a service statement. 
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1 75. A method according to Claim 70, further comprising 

2 the step of making additional requests to change the 

3 requested supplemental information. 

1 76. A method according to Claim 70, further comprising 

2 the step of initiating a computer application program for 

3 processing the request for supplemental information made 

4 in said requesting step. 

1 77. A system, comprising: 

2 a data processing apparatus including a CPU and 

3 a memory operatively connected to said CPU; 

4 said memory containing a first program, adapted 

5 to be executed by said CPU, for requesting supplemental 

6 information related to a video program, for transmitting 

7 synchronization information related to the video program, 

8 for receiving the requested supplemental information via 

9 a data communication network, and for displaying the 
10 supplemental information; 

U a server including a controller and a storage 

12 device operatively connected to said controller; and 

13 said storage device contains a second program, adapted to 

14 be executed by the controller, for processing the request 

15 for supplemental information, for receiving 

16 synchronization information, for synchronizing the 

17 supplemental information to the video program, and for 

18 transmitting the synchronized supplemental information 
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19 through the data communication network to said data 

20 processing system. 

1 78. A method for providing supplemental information 

2 related to a video program using a data processing 

3 apparatus including a CPU and a memory operative ly 

4 connected to the CPU and containing a first program 

5 adapted to be executed by the CPU for requesting the 

6 supplemental information, and a server including a 

7 controller and a storage device operatively connected to 

8 the controller containing a second program adapted to be 

9 executed by the controller for processing the request for 

10 the supplemental inf ormation, comprising the steps of: 

11 initiating the first program; 

12 requesting the supplemental information; 

13 transmitting synchronization information related to the 

14 video program to the server; 

15 processing the request for the supplemental 

16 information by having the controller in the server 

17 execute the second program; 

18 synchronizing the requested supplemental 

19 information to the video program using the 

20 synchronization information; 

21 receiving, by the data processing apparatus, 

22 the synchronized supplemental information; and 

23 displaying, by the data processing apparatus, 

24 the synchronized supplemental information. 
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! 79. A method for receiving supplemental information, 

2 comprising the steps of : 

3 initiating a computer program; 

4 receiving program identification information 

5 and synchronization information related to a video 

6 program; 

7 processing the program identification 

8 information and the synchronization information; 

9 j receiving the supplemental information 

10 synchronized to the events and action of the video 

11 program; and 

12 displaying the supplemental information. 

1 80. A system, comprising: 

2 ; a display device configured to receive a video 

3 program, said video program including ah audio component, 

4 a visual component and synchronization information; 

5 . .a server including a controller and a storage 

6 device operatively connected to said controller; 

7 said storage device contains a first program, adapted to 

8 be executed by said controller, for processing a request 

9 for supplemental information related to said video 

10 program, for receiving information corresponding to said 

11 synchronization information, for synchronizing the 

12 supplemental information to said video program, and for 

13 transmitting the synchronized supplemental information . 

14 through a data communication network; 
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15 a data processing apparatus including a CPU and 

16 a memory operatively connected to said CPU; and 

17 said memory containing a second program, 



18 adapted to be executed by said CPU, for requesting the 

19 supplemental information from said server, for 

20 transmitting information corresponding to said 

21 synchronization information to said server, for receiving 

22 the supplemental information via the data communication 

23 network, and for displaying the supplemental information 

24 synchronized to said video program. 



1 -81. A method for providing supplemental information 

2 related to a video program using a display device for 

3 displaying the video program, a data processing apparatus 

4 ; including a CPU and memory operatively connected to the 

5 CPU and containing a first program adapted to be executed 

6 : by the CPU for requesting the supplemental information, 

7 and a server including a controller and a storage device 

8 . operatively connected to the controller containing a 

9 second program adapted to be executed by the controller 
10 for processing the requests, comprising the steps of: 



11 displaying the video program on the display 

12 device, where the video program includes an audio 

13 component, a visual component and synchronization 

14 information; 

15 initiating the first program; 

■W requesting the supplemental information; 
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17 transmitting information corresponding to the 

18 synchronization information from the video program to the 

19 server; 

20 processing the request for the supplemental 

21 information by having the controller in the server 

22 execute the second program; 

23 synchronizing the requested supplemental 

24 information to the video program using the information 

25 transmitted in said transmitting step; 

26 receiving, by the data processing apparatus, 

27 the synchronized supplemental information; and 

28 displaying, by the data processing apparatus, 

29 the synchronized supplemental information. 

1 82. An article, comprising: 

. 2 a recording medium having recorded thereon a visual 

3 component and an audio component for display on at least 

4 one display device; and 

5 synchronization information recorded on said recording • 

6 medium for display on at least one display device, said 

7 synchronization information adapted to synchronize 

8 supplemental visual and audio information not recorded on 

9 said recording medium to said visual and audio 
10 components. 



67 



CA 02283940 1999-09-13 



WO 98/43406 PCT/US98/05112 

1 83. The article according to Claim 82, wherein said 

2 synchronization information comprises a time-code changed 

3 at predetermined intervals. 

1 84. A signal containing information for requesting 

2 supplemental information relating to broadcast program 

3 information conveyed by the signal, comprising: 

4 a broadcast program including a visual component arid an 

5 audio component for transmission to at least one display 

6 device; and 

7 program identification information and 

8 synchronization information included as part of said 

9 video component, said program identification information 

10 including a code for identifying said program, said 

11 synchronization information for synchronizing the. 

12 supplemental information with the said program, 

13 wherein said synchronization information is 

14 representative of the amount of time elapsed from the 

15 start of said broadcast program. 
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